Skip to content

fix(wizard-ask): let one call carry a whole connector form - #1319

Merged
Gilbert09 merged 1 commit into
mainfrom
posthog/ask-cap-fits-connector-form
Sep 23, 2026
Merged

Gilbert09 merged 1 commit into
mainfrom
posthog/ask-cap-fits-connector-form

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

The seeded data-warehouse step collects a source's connection details through
wizard_ask, and both harness facades capped one call at eight questions. That
is below what the connectors advertise: the widest source forms carry nine
fields, and several of the databases this step is most often offered for sit at
exactly eight, leaving no room for a confirmation or a corrected retry. A form
with an SSH tunnel expands further still, because the tunnel field is a
composite.

The guidance around the tool reads as "one subject, one call", and nothing told
the agent that a subject too wide for one call may simply be split. So a source
the wizard could have connected in the terminal was instead reported as
uncollectable and handed back as a link for the user to follow themselves — the
exact hand-off the in-CLI setup exists to avoid.

Changes

  • One shared ASK_MAX_QUESTIONS_PER_CALL, sized to carry the widest connector
    form with a little headroom, replaces the literal 8 in the MCP facade's zod
    schema and the pi facade's typebox mirror. The orchestrator runs on pi, so
    keeping the two level matters.
  • The tool description and the batching nudge quote the constant instead of
    hard-coding a number, so the limit and what the agent is told about it cannot
    drift apart.
  • The description now states what to do with a subject wider than one call:
    collect it over consecutive calls reusing the same subject — which the
    per-subject batching guard already permits — and never treat the width as a
    reason to fall back to a link.
  • Docs line in the self-driving architecture note updated to match.

No change to the per-run call cap, the adjacency threshold, the refund
behaviour, or any analytics property.

Test plan

  • pnpm typecheck, npx vitest run (3209 tests, 184 files, all passing), and
    pnpm lint (0 errors) on the full tree.
  • New tests: the constant covers the widest advertised connector form; the tool
    description quotes it and carries the split-don't-abandon instruction; the
    nudge quotes it; and the pi schema — the one actually in effect for the
    orchestrator — caps at the shared constant rather than a harness-local number.

Created with PostHog Desktop

🤖 Generated with Claude Code

The per-call question limit was eight, below the widest data-warehouse
connector's advertised field set. The databases this step is mostly offered
for sat at or over it, so the agent could not collect one in the single call
the guidance asks for, and handed the user a browser link instead.

Both harness facades now read one shared ASK_MAX_QUESTIONS_PER_CALL, and the
tool description and the batching nudge quote it, so the limit and what the
agent is told about it cannot drift. The description also says what to do with
a subject wider than one call: split it over consecutive calls reusing the same
`subject`, which the batching guard already allows, rather than abandoning it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: 4c9a85df-9b83-4a2f-9a8e-6a127402705f
Copilot AI lite review requested due to automatic review settings September 23, 2026 13:09
@Gilbert09
Gilbert09 requested review from a team as code owners September 23, 2026 13:09
@Gilbert09 Gilbert09 self-assigned this Sep 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@Gilbert09 Gilbert09 added the stamphog Auto-merge stamp for automated PRs label Sep 23, 2026 — with PostHog
@Gilbert09
Gilbert09 merged commit fde4fac into main Sep 23, 2026
21 checks passed
@Gilbert09
Gilbert09 deleted the posthog/ask-cap-fits-connector-form branch September 23, 2026 19:23
gewenyu99 added a commit that referenced this pull request Sep 23, 2026
Brings in main (#1319). No conflicts.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
gewenyu99 added a commit that referenced this pull request Sep 24, 2026
Carries the final Release A branch, which now includes main's #1319,
#1235, #1305 and 2.77.0. Tests the merge added under src/lib/programs
move to src/programs with B1 import paths.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
gewenyu99 added a commit that referenced this pull request Sep 24, 2026
Carries main's #1319 and #1235 through the callable host:
- runProgram loads wizard flags before it resolves the run, so the
  integration prompt reads the AI Observability and Logs kill switch.
  artifacts.reportFile is now set just before the agent starts, so a
  pre-run failure leaves it absent.
- One excludedIntegrationTaskTypes feeds both program registries.
- The legacy adapter records task outcomes for the e2e harness, and
  TASK_OUTCOMES_KEY lives in runner/shared/types to stay out of the
  startup closure.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Auto-merge stamp for automated PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants